TU Delft expert judgment data base

نویسندگان

  • Roger M. Cooke
  • Louis H. J. Goossens
چکیده

We review the applications of structured expert judgment uncertainty quantification using the "classical model" developed at the Delft University of Technology over the last 17 years (Cooke, 1991). These involve 45 expert panels, performed under contract with problem owners who reviewed and approved the results. With a few exceptions, all these applications involved the use of seed variables; that is, variables from the experts' area of expertise for which the true values are available post hoc. Seed variables are used to (1) measure expert performance, (2) enable performance based weighted combination of experts' distributions, and (3) evaluate and hopefully validate the resulting combination or "decision maker". This article reviews the classical model for structured expert judgment and the performance measures, reviews applications, comparing performance based decision makers with "equal weight" decision makers, and collects some lessons learned. Acknowledgement: The authors gratefully acknowledge the contributions of many people who cooperated in developing this database. Willy Aspinall, and Tim Bedford are independently responsible for a quarter of the studies. Introduction The pro’s and con’s of different weighting schemes remain a subject of research. The European Union contracted the TU Delft to review its applications both within EU projects, and elsewhere, in which experts assessed variables in their field of expertise for which the true values are known, in addition to variables of interest (Goossens et al 1996, 1998). These are called seed, or calibration, variables. Since then, the TU Delft expert judgment data base has nearly doubled. We now have studies involving over 67,000 experts' subjective probability distributions. The main sectors and summary information are given in Table 1 below. Table 1 Summary of applications per sector Sector # of experts # of variables # of elicitations Nuclear applications 98 2,203 20,461 Chemical ind. & gas industry 56 403 4,491 Groundwater / water pollution / dike ring / barriers 49 212 3,714 Aerospace sector / space debris /aviation 51 161 1,149

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Computer Assisted Study Planning

Automated Study Planning (ASP) system was designed to assist the students in organising their studies. Interactive expert system ASP forms a part of a bigger project that concerns computer-based education and training in higher learning [1]. The formulation of the study-planning process, whereupon the reasoning engine of ASP is built, has been acquired from the practical knowledge of the study-...

متن کامل

Expert Aggregation with Dependence

The measure for expert dependence proposed by Jouini and Clemen (clemen) is implemented for expert judgement data gathered at the T.U. Delft. Experts show less dependence than might have been supposed, though more sensitive measures might reveal more. Clemen’s copula for aggregation is implemented and performance is compared with performance-based combinations for two illustrative cases.

متن کامل

Ranking the university stars: are opinions sufficient?

(USA), are the only global university performance tables to judge world-class universities across all of their core mis-sions—teaching, research, knowledge transfer and international status. The top university rankings employ 13 carefully calibrated performance indicators to provide the most comprehensive and balanced comparisons available, which are trusted by students, academics, university l...

متن کامل

On the performance of social network and likelihood-based expert weighting schemes

Using expert judgment data from the TU Delft's expert judgment data base, we compare the performance of different weighting schemes, namely equal weighting, performance based weighting from the classical model (Cooke, 1991), social network weighting and likelihood weighting. The picture that emerges with regard to social network weights is rather mixed. SN theory does not provide an alternative...

متن کامل

CWI and TU Delft Notebook TREC 2013: Contextual Suggestion, Federated Web Search, KBA, and Web Tracks

This paper provides an overview of the work done at the Centrum Wiskunde & Informatica (CWI) and Delft University of Technology (TU Delft) for different tracks of TREC 2013. We participated in the Contextual Suggestion Track, the Federated Web Search Track, the Knowledge Base Acceleration (KBA) Track, and the Web Ad-hoc Track. In the Contextual Suggestion track, we focused on filtering the enti...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:
  • Rel. Eng. & Sys. Safety

دوره 93  شماره 

صفحات  -

تاریخ انتشار 2008